Depth sensing is the determination of the distance between a known point in three dimensional (“3D”) space (e.g., a sensor) and a point of interest (“POI”) on a surface of an object. Depth sensing is also known as texture sensing because determining the respective distances of a plurality of POIs on a surface determines the texture of that surface. Depth or texture sensing is useful for many computer vision systems, including mixed reality systems.
Modern computing and display technologies have facilitated the development of mixed reality systems for so called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR”, scenario typically involves presentation of digital or virtual image information without transparency to actual real-world visual input. An augmented reality, or “AR”, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user (i.e., transparency to other actual real-world visual input). Accordingly, AR scenarios involve presentation of digital or virtual image information with transparency to other actual real-world visual input.
Various optical systems generate images at various depths for displaying mixed reality (VR and AR) scenarios. Some such optical systems are described in U.S. Utility patent application Ser. No. 14/738,877, the contents of which have been previously incorporated-by-reference herein. Other such optical systems for displaying mixed reality scenarios are described in U.S. Utility patent application Ser. No. 14/555,585 filed on Nov. 27, 2014, the contents of which are hereby expressly and fully incorporated by reference in their entirety, as though set forth in full.
AR scenarios often include presentation of virtual image elements in relationship to real-world objects. For example, referring to FIG. 1, an augmented reality scene 100 is depicted wherein a user of an AR technology sees a real-world park-like setting 102 featuring people, trees, buildings in the background, and a concrete platform 104. In addition to these items, the user of the AR technology also perceives that he “sees” a robot statue 106 standing upon the real-world platform 104, and a cartoon-like avatar character 108 flying by which seems to be a personification of a bumble bee, even though these elements 106, 108 do not exist in the real world. In order to present a believable or passable AR scene 100, the depth of real world objects (e.g., the platform 104) must be determined to present virtual objects (e.g., the robot statue 106) in relation to real world objects.
VR scenarios that include reproduction of portions of real world environments can also benefit from determination of the depth and texture of those portions of the real world environment. Accurate depth and texture information will result in more accurate VR scenarios. Both AR and VR scenarios may also include outwardly directed cameras to capture portions of real world environments (e.g., for analysis or transmission). Focusing these outwardly directed cameras can be aided by determination of the depth of those portions of the real world environment.
One approach to depth sensing includes measuring the respective angles between the optical axes of two images (which are separated by a known distance at a known orientation) of a single POI on a surface and the POI on the respective images. Then determining the depth of the surface by triangulating the measured angles and the known distance between the image capture locations. Problems with this approach include (1) identification of the POI (especially on a homogenous surface) (“identification problem”) in the first image and (2) identification of the corresponding POI in the second image (“correspondence problem”). The systems and methods described herein are configured to address these challenges.